The present invention relates to software tools for testing and monitoring the operation of web-based and other transactional servers.
A variety of commercially-available software tools exist for assisting companies in testing the performance and functionality of their web-based transactional servers and associated applications prior to deployment. Examples of such tools include the LoadRunner(copyright), WinRunner(copyright) and Astra QuickTest(copyright) products of Mercury Interactive Corporation, the assignee of the present application.
Using these products, a user can record or otherwise create a test script which specifies a sequence of user interactions with the transactional server. The user may also optionally specify certain expected responses from the transactional server, which may be added to the test script as verification points. For example, the user may record a session with a web-based travel reservation system during which the user searches for a particular flight, and may then define one or more verification points to check for an expected flight number, departure time or ticket price.
Test scripts generated through this process are xe2x80x9cplayedxe2x80x9d or xe2x80x9cexecutedxe2x80x9d to simulate the actions of usersxe2x80x94typically prior to deployment of the component being tested. During this process, the testing tool monitors the performance of the transactional server, including determining the pass/fail status of any verification points. Multiple test scripts may be replayed concurrently to simulate the load of a large number of users. Using an automation interface of the LoadRunner product, it is possible to dispatch test scripts to remote computers for execution.
The results of the test are typically communicated to the user through a series of reports that are accessible through the user interface of the testing tool. The reports may contain, for example, graphs or charts of the observed response times for various types of transactions. Performance problems discovered through the testing process may be corrected by programmers or system administrators.
A variety of tools and services also exist that allow web site operators to monitor the post-deployment performance of their web sites. For example, Keynote Systems Inc. of San Mateo Calif. provides a service which uses automated agents to access a web site at regular intervals throughout the day. The agents computers, which are provided by Keynote Systems in selected major cities, measure the time required to perform various web site functions, and report the results to a server provided by Keynote Systems. The owner or operator of the web site can access this server using a web browser to view the collected performance data on a city-by-city or other basis. Other types of existing monitoring tools include log analysis tools that process access logs generated by web servers, and packet sniffing tools that monitor traffic to and from the web server.
One aspect of the invention is a system for monitoring a server from a plurality of locations on a computer network. The system comprises a plurality of agent computers coupled to a computer network, each of the agent computers having an agent component installed thereon. The agent component includes functionality for accessing server systems over the computer network as simulated users, and for monitoring and reporting performance of such server systems. The system also includes a controller component that provides functionality for a user to define a monitoring session in which a selected group of the agent computers access and monitor a target server system. The controller component includes a user interface through which the user can select individual agent computers from said plurality of agent computers to include in the group, and individually assign to each selected agent computer one or more transactions that specify simulated user actions. The user of the controller component can thereby define a monitoring session in which different agent computers simulate different user interactions with the target server system.
Another aspect of the invention is a computer-implemented method of facilitating creation by a user of a monitoring session in which agent computers access and monitor a target server system over a computer network. The method comprises providing a user interface through which the user may select a group of agent computers to use for the monitoring session, and displaying the agent computers selected by the user as respective nodes within an expandable tree representation of the monitoring session. The method further includes presenting to the user a display of transactions, and an option to selectively assign one or more of the transactions to each agent computer in the group, wherein each transaction specifies one or more request messages for accessing the target server as a simulated user. In response to user assignment of a transaction to a selected agent computer, the transaction is displayed within the expandable tree representation as a child of the selected agent computer.
The present invention also includes an agent component that is adapted to run on a networked computer to remotely monitor operation of a server system. The agent component is configured to execute a transaction as a simulated user of the server system, and to store a sequence of screens displays returned by the transactional server during execution of the transaction. The agent component also determines whether the transaction completed successfully, and if it did not, forwards the sequence of screen displays to a repository for subsequent viewing.
The invention also includes a method of facilitating user analysis of operation of a transactional server. The method comprises collecting performance data generated by a plurality of agent computers that are programmed to access the transactional server as simulated users that execute transactions, said performance data including transaction pass/fail data, and including screen displays returned by the transactional server. The performance data, including the transaction pass/fail data, is displayed within an online report that provides a user option to drill down to view a sequence of said screen displays that correspond to a transaction failure. The user can thereby view a sequence of server responses associated with the transaction failure.
The invention further includes an automated method of monitoring operation of a server system as seen from an agent computer at a remote location. The method comprises, on the agent computer, accessing the server system as a simulated user while monitoring a path delay between the agent computer and the server system. When the path delay satisfies a pre-specified condition, a delay along each segment of a network path between the agent computer and the server system is measured to generate a plurality of segment delay measurements. The segment delay measurements are reported to a central monitoring component that receives data reported by a plurality of agent computers.
Another aspect of the invention is a system for monitoring performance of a server system. The system comprises a plurality of agent computers that access the server system over a computer network as simulated users and report performance data resulting therefrom, wherein the agent computers have assigned thereto attributes from each of a plurality of attribute categories. The system also includes a reports server that collects the performance data reported by the plurality of agent computers, and displays the performance data in an aggregated form. The reports server (1) provides a user option to specify an attribute set that includes attributes from each of the plurality of attribute categories, whereby a user may specify an attribute-based subset of the plurality of agent computers, and (2) generates an attribute-filtered view of the performance data indicative of how the server system is collectively seen by the attribute-based subset of agent computers.